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1 . A neural network system comprising: 

a feedforward network comprising at least one neuron circuit for producing an 
activation function and a first derivative of the activation function; 

a weight updating circuit for producing updated weights to the feedforward 
network; and 

an error back-propagation network for receiving the first derivative of the 
activation function and to provide weight change data information to the weight 
updating circuit. 

2. The system of claim 1 , wherein the neuron circuit comprises: 
a linear resistor circuit; 

a first differential circuit electrically connected to the linear resistor circuit, the 
first differential circuit configured to output a first differential circuit output; 

a second differential circuit electrically connected to the linear resistor circuit, 
the second differential circuit configured to output a second differential circuit output; 
and 

wherein the first differential circuit output is the activation function and the first 
differential circuit output minus the second differential circuit output is the first 
derivative of the activation function. 
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3. The system of claim 2, wherein the linear resistor circuit comprises a 
first linear resistor circuit transistor and a second linear resistor circuit transistor 
configured in a differential transistor pair. 

4. The system of claim 3, further comprising a first reference voltage 
configured to cause the first linear resistor circuit transistor and the second linear 
resistor circuit transistor to operate in a linear range. 

5. The system of claim 4, wherein the first differential circuit output is 
shifted when the magnitude of the first reference voltage is varied. 

6. The system of claim 3, wherein the linear resistor circuit is controllable 
by a first linear resistor circuit transistor gate voltage and a second linear resistor 
circuit transistor the gate voltage. 

7. The system of claim 6, wherein a gain factor of the first differential 
circuit output is varied when the magnitude of the first linear resistor circuit transistor 
gate voltage and the second linear resistor circuit transistor the gate voltage are 
varied. 

8. The system of claim 7, wherein saturation levels of the activation 
function remain constant for different gain factor values. 



17 



FINNEGAN 
HENDERSON 
FAR ABOW 
GARRETTSc 
DUNNER kif 

1300 I Street, NW 
Washington, DC 20005 
202.408.4000 
Fax 202.408.4400 
www.finnegan.com 



9. The system of claim 2, wherein: 

the first differential circuit comprises a first differential transistor pair having a 
first differential transistor pair first port electrically connected to a linear resistor 
circuit output and a first differential transistor pair second port electrically connected 
to a second reference voltage; and 

the second differential circuit comprises a second differential transistor pair 
having a second differential transistor pair first port electrically connected to the 
linear resistor circuit output and a second differential transistor pair second port 
electrically connected to a third reference voltage, the third reference voltage being 
less than the second reference voltage by a fixed amount. 

1 0. The system of claim 2, wherein the error between a fitted sigmoid 
function and the first differential circuit output is less than 3%. 

1 1 . The system of claim 2, wherein the error between the first derivative of 
a fitted sigmoid function and the first differential circuit output minus the second 
differential circuit output is less than 5%. 

1 2. The system of claim 1 , wherein the neural network system is 
constructed on one electronic chip. 
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1 3. A method for establishing a neural network comprising: 

producing an activation function and a first derivative of the activation function 
utilizing at least one neuron circuit in a feedforward network; 

providing updated weights to the feedforward network utilizing a weight 
updating circuit; 

receiving the first derivative of the activation function by an error back- 
propagation network; and 

providing weight change data information to the weight updating circuit from 
the error back-propagation network. 

14. The method of claim 13, wherein the neuron circuit comprises: 
a linear resistor circuit; 

a first differential circuit electrically connected to the linear resistor circuit, the 
first differential circuit configured to output a first differential circuit output; 

a second differential circuit electrically connected to the linear resistor circuit, 
the second differential circuit configured to output a second differential circuit output; 
and 

wherein the first differential circuit output is the activation function and the first 
differential circuit output minus the second differential circuit output is the first 
derivative of the activation function. 
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15. The method of claim 14, wherein the linear resistor circuit comprises a 
first linear resistor circuit transistor and a second linear resistor circuit transistor 
configured in a differential transistor pair. 

16. The method of claim 1 5, further comprising a first reference voltage 
configured to cause the first linear resistor circuit transistor and the second linear 
resistor circuit transistor to operate in a linear range. 

1 7. The method of claim 16, wherein the first differential circuit output is 
shifted when the magnitude of the first reference voltage is varied. 

1 8. The method of claim 1 5, wherein the linear resistor circuit is 
controllable by a first linear resistor circuit transistor gate voltage and a second linear 
resistor circuit transistor the gate voltage. 

1 9. The method of claim 18, wherein a gain factor of the first differential 
circuit output is varied when the magnitude of the first linear resistor circuit transistor 
gate voltage and the second linear resistor circuit transistor the gate voltage are 
varied. 

20. The method of claim 19, wherein saturation levels of the activation 
function remain constant for different gain factor values. 
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21 . The method of claim 14, wherein: 

the first differential circuit comprises a first differential transistor pair having a 
first differential transistor pair first port electrically connected to a linear resistor 
circuit output and a first differential transistor pair second port electrically connected 
to a second reference voltage; and 

the second differential circuit comprises a second differential transistor pair 
having a second differential transistor pair first port electrically connected to the 
linear resistor circuit output and a second differential transistor pair second port 
electrically connected to a third reference voltage, the third reference voltage being 
less than the second reference voltage by a fixed amount. 

22. The system of claim 14, wherein the error between a fitted sigmoid 
function and the first differential circuit output is less than 3%. 

23. The system of claim 14, wherein the error between the first derivative 
of a fitted sigmoid function and the first differential circuit output minus the second 
differential circuit output is less than 5%. 

24. The system of claim 13, wherein the neural network system is 
constructed on one electronic chip. 
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25. A neural network system comprising: 

a feedforward network comprising at least one neuron circuit for producing an 
activation function and a first derivative of the activation function, the at least one 
neuron circuit comprising: 

a linear resistor circuit comprising, 

a first linear resistor circuit transistor and a second linear 
resistor circuit transistor configured in a differential transistor pair, and 

a first reference voltage configured to cause the first linear 
resistor circuit transistor and the second linear resistor circuit transistor 
to operate in a linear range, 

a first differential circuit electrically connected to the linear resistor 
circuit, the first differential circuit configured to output a first differential circuit 
output, the first differential circuit comprises a first differential transistor pair 
having a first differential transistor pair first port electrically connected to a 
linear resistor circuit output and a first differential transistor pair second port 
electrically connected to a second reference voltage, 

a second differential circuit electrically connected to the linear resistor 
circuit, the second differential circuit configured to output a second differential 
circuit output, the second differential circuit comprises a second differential 
transistor pair having a second differential transistor pair first port electrically 
connected to the linear resistor circuit output and a second differential 
transistor pair second port electrically connected to a third reference voltage, 
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the third reference voltage being less than the second reference voltage by a 
fixed amount, and 

wherein the first differential circuit output is the activation function and 
the first differential circuit output minus the second differential circuit output is 
the first derivative of the activation function, the error between a fitted sigmoid 
function and the first differential circuit output is less than 3%, the error 
between the first derivative of a fitted sigmoid function and the first differential 
circuit output minus the second differential circuit output is less than 5%, the 
first differential circuit output is shifted when the magnitude of the first 
reference voltage is varied, the linear resistor circuit is controllable by a first 
linear resistor circuit transistor gate voltage and a second linear resistor circuit 
transistor the gate voltage, a gain factor of the first differential circuit output is 
varied when the magnitude of the first linear resistor circuit transistor gate 
voltage and the second linear resistor circuit transistor the gate voltage are 
varied, and saturation levels of the activation function remain constant for 
different gain factor values; 

a weight updating circuit for producing updated weights to the feedforward 
network; 

an error back-propagation network for receiving the first derivative of the 
activation function and to provide weight change data information to the weight 
updating circuit; and 

wherein the neural network system is constructed on one electronic chip. 
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